Rate of convergence of penalized likelihood context tree estimators

The Bayesian Information Criterion (BIC) was first proposed by Schwarz (1978) as a model selection technique. It was thought that BIC was not appropriate for the case of context tree estimation, because of the huge number of trees that has to be tested. Recently, Csiszár & Talata (2006) proved the almost surely consistency of the BIC estimator and they also showed that it can be computed in linear time. Nevertheless, the rate of convergence of the BIC estimator remained as an open question, even in the simpler form for estimating the order of a Markov chain. The latter is a particular case of the tree estimation problem and can be derived from our results. The paper is organized as follows. In Section 2 we introduce some definitions and state the main results. In Section 3 we obtain exponential inequalities for empirical counts and empirical transition probabilities, generalizing a previous result in Galves et al. (2006) to the case of unbounded trees. These results are the key in the proof of the rate of convergence of the BIC estimator and by their relevant importance we included them in a separate section. Finally, in Section 4 we proved the main results in this paper and in Section 5 we present some discussion.